Image processing apparatus, method, and program

ABSTRACT

An image processing apparatus, a method, and a program for allowing cells to be quantitatively observed. A computer obtains a cell membrane image obtained by performing fluorescent observation on a cell membrane of a cell serving as a sample and a tricellular tight junction (tTJ) image obtained by performing fluorescent observation on a protein localized in a tTJ of the cell. The computer derives the size of area of a region of the cell by identifying the region of each cell from the cell membrane image, derives the size of area of the region of the protein localized in the cell from the tTJ image, and dividing the obtained size of area of the region of the protein by the size of area of the region of the cell, thus calculating an index of adhesion strength of the cells. The invention can be applied to an observation system.

This is a Continuation of application Ser. No. 14/103,040 filed Dec. 11,2013 (now U.S. Pat. No. 9,280,698), which in turn is a continuation ofPCT/JP2012/065684 filed Jun. 20, 2012, which claims foreign priority toJP 2011-135959 filed Jun. 20, 2011. The disclosures of the priorapplications are hereby incorporated by reference herein in theirentirety.

TECHNICAL FIELD

The present invention relates to an image processing apparatus, amethod, and a program for allowing cells to be quantitatively observed.

BACKGROUND ART

In general, various genes are related to human diseases and conditions,and in particular, for diseases of which cause is unknown, a method forfinding related genes by using a technique such as gene screening methodfor knocking out the function of the related genes under many conditionsand solving the diseases is sought for.

The effect of medicine in such screening is known to appear mostsignificantly in change of localization of protein in a cell. Therefore,a method for quantitatively evaluating localization of protein accordingto individual screening is sought for, and for example, high contentscreening systems for performing analysis from fluorescence microscopeimages of cells have been developed.

In order to perform such screening, it is necessary to take an image ofa culture vessel capable of cultivation under many conditions such as amicrowell plate. An apparatus integrally including a microscope andanalysis software is also known (for example, see Patent Literature 1).In high content screening performed by this apparatus, in a region of acell nucleus which is segmented in image processing, a designated regioncentered at a predetermined point is adopted as a region of analysistarget, which is analyzed.

By the way, a living body is divided into various sections byepitheliums, and a cell adhesion structure existing in each section hasa barrier function for controlling passage of substance between cellsand organs. The cell adhesion structure is important for maintaining aform of organs such as blood vessels, and is also useful for maintainingthe environment in the ecology.

The abnormality of the barrier function may cause various diseasesconcerning skin, liver, kidney, and the like, and it has been revealedthat it is also related to invasion of cancer and passage of white bloodcells, and clarifying the mechanism of the cell adhesion structure isattracting attention.

With recent studies, protein condensed in tTJ (tricellular tightjunction) which is a portion where three cells are in contact with eachother has been discovered, and it is suggested that this plays animportant role in the intensity of cell adhesion. For example, theprotein condensed in tTJ includes tricellulin, LSR (Lipolysis stimulatedlipoprotein receptor), and the like.

It is desired to find the detailed function of the protein condensed insuch tTJ and develop evaluation method. More specifically, for example,a method and the like is desired to quantitatively evaluate thephenomenon that the intensity of cell adhesion is reduced because ofchanges of the localization of the protein existing in tTJ by causingmedicine to take effect.

CITATION LIST Patent Literature

Patent Literature 1: JP 2010-527007 W

SUMMARY OF INVENTION Technical Problem

However, with the above technique, it used to be difficult toquantitatively observe a cell of observation target such as adhesion ofthe cell and localization of protein condensed in tTJ. For example, withthe technique disclosed in Patent Literature 1, it is impossible toevaluate localization of complicated protein such as protein related tocell adhesion, and therefore, there was no choice but to determine theeffect of medicine with the eyes of a person.

The present invention is made in view of such circumstances, and enablesa cell to be quantitatively observed.

Solution to Problem

An image processing apparatus of the present invention includes firstimage processing means configured to identify, on the basis of a firstobservation image indicating localization of a protein related to apredetermined feature of a cell of observation target, a region of theprotein in the first observation image, and calculation means configuredto calculate, on the basis of quantitative information related to theregion of the protein identified, an index representing the feature.

Advantageous Effects of Invention

According to the present invention, a cell can be quantitativelyobserved.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a figure illustrating an example of configuration of anembodiment of an observation system to which the present invention isapplied.

FIG. 2 is a figure illustrating an example of configuration of acomputer.

FIG. 3 is a flowchart for explaining observation processing.

FIG. 4 is a figure illustrating an example of a cell membrane image anda tTJ image.

FIG. 5 is a figure for explaining generation of a cell mask image,

FIG. 6 is a figure for explaining generation of an expansion mask image.

FIG. 7 is a figure illustrating an example of processing result of theindex of adhesion strength of cells.

FIG. 8 is a figure illustrating an example of processing result of anindex of adhesion strength of a cell.

DESCRIPTION OF EMBODIMENTS

Hereinafter, an embodiment to which the present invention is appliedwill be explained with reference to drawings.

[Example of Configuration of Observation System]

FIG. 1 is a figure illustrating an example of configuration of anembodiment of an observation system to which the present invention isapplied.

This observation system is to quantitatively observe a cell ofobservation target. Hereinafter, for example, explanation will continuein a case where epithelium cells of a living body are observationtarget, and adhesion strength of the cells is quantitatively observed.

In this case, the observation system is a screening system forperforming analysis for determining the strength of adhesion of thecells, and in the screening, the index of cell adhesion strength of eachcell is calculated from an image obtained by visualizing the celladhesion molecules.

The observation system of FIG. 1 includes a light source 11, a beamexpander 12, a scan head 13, a microscope 14, a controller 15, and acomputer 16, and a sample 17 as observation target is observed withfluorescence by the observation system. In this case, the sample 17 isepithelium cells. Multiple epithelium cells are put in a container suchas a microwell plate, and observed.

In the observation system, the scan head 13 for scanning light isattached to the microscope 14. The scan head 13 and the microscope 14function as a two-photon microscope-attached confocal microscope(hereinafter simply referred to as confocal microscope). Morespecifically, this confocal microscope realizes confocal observationbased on ordinary one-photon excitation method and observation based ontwo-photon excitation method.

The light source 11 emits illumination light (excitation light) toirradiate the sample 17 as observation target. For example, when thesample 17 is observed according to the two-photon excitation method, ashort pulsed laser light source and the like is used as the light source11.

The illumination light which is emitted from the light source 11 isshaped by the beam expander 12 so that the beam diameter is enlarged,and made into parallel light beam, which is incident upon the scan head13.

The illumination light which has been led to the scan head 13 by spacepropagation as described above is incident upon the microscope 14 viathe dichroic mirror 31, scan unit 32, and relay lens 33 provided in thescan head 13. Then, the sample 17 placed on the stage 36 is irradiatedwith the illumination light incident upon the microscope 14 via thedichroic mirror 34 and the object lens 35 in the microscope 14.

At this occasion, the scan unit 32 deflects the illumination lightincident from the dichroic mirror 31 in the horizontal and depthwisedirection in the figure, thus scanning the illumination light on thesample 17. When the sample 17 is irradiated with the illumination light,fluorescence which becomes the observation light is generated from thesample 17, and this fluorescence is incident upon the dichroic mirror 34via the object lens 35.

In the case of two-photon excitation method, the fluorescence isgenerated only in proximity to the focal point, and high spatialresolution is provided by itself, and no pinhole is necessary.Therefore, in order to detect fluorescence with a high degree ofefficiency, a detection device can be configured to be provided at aposition as close to the sample as possible.

In such configuration, the dichroic mirror 34 reflects the fluorescencefrom the object lens 35 as necessary, so that it is incident upon thecondensing lens 37. More specifically, the dichroic mirror 34 isconfigured to move physically, and when the fluorescence generated fromthe sample 17 by the two-photon excitation method is observed, thedichroic mirror 34 is arranged in an optical path of the illuminationlight (detection subject optical path), and when the fluorescence isobserved with confocal method by ordinary one-photon excitation method,it is not arranged in the detection subject optical path.

When the dichroic mirror 34 is arranged in the detection subject opticalpath, the dichroic mirror 34 transmits the illumination light from therelay lens 33, and reflects the fluorescence from the sample 17, so thatit is incident upon the condensing lens 37.

The fluorescence incident upon the condensing lens 37 is condensed bythe condensing lens 37, and is received by the photoelectric detectiondevice 38. The photoelectric detection device 38 performs photoelectricconversion on the received fluorescence. Then, an electric signalobtained from the photoelectric conversion is provided to the computer16 from the photoelectric detection device 38 via the controller 15.

On the other hand, when the dichroic mirror 34 is not arranged in thedetection subject optical path, the sample 17 is irradiated with theillumination light from the relay lens 33 via the object lens 35. Then,the fluorescence from the sample 17 passes the optical path of theillumination light in the opposite direction and is incident upon thedichroic mirror 31. More specifically, the fluorescence incident uponthe object lens 35 from the sample 17 is incident upon the dichroicmirror 31 via the relay lens 33 and the scan unit 32.

The dichroic mirror 31 reflects the fluorescence from the scan unit 32and it is incident upon the condensing lens 39. This dichroic mirror 31passes the light having the wavelength of the illumination light, andreflects the light having the wavelength of the fluorescence.

The fluorescence incident upon the condensing lens 39 from the dichroicmirror 31 is condensed by the condensing lens 39, and passes through apinhole provided in a confocal diaphragm 40, and is received by thephotoelectric detection device 41. Then, the electric signal obtained byperforming photoelectric conversion on the fluorescence received by thephotoelectric detection device 41 is provided from the photoelectricdetection device 41 via the controller 15 to the computer 16.

As described above, in the case of the confocal observation according tothe one-photon excitation method, the fluorescence takes the passagefrom the object lens 35 to the photoelectric detection device 41. In thecase of the observation according to the two-photon excitation method,detection can be done based on this passage by opening the pinhole.Hereinafter, for example, explanation will continue in a case where thesample 17 is observed according to the two-photon excitation method.

On the basis of the electric signal provided from the photoelectricdetection device 41 or the photoelectric detection device 38 via thecontroller 15, the computer 16 generates an image of the sample 17, andon the basis of the generated image, the computer 16 calculates an indexAD of adhesion strength of the cells for the sample 17.

For example, the observation system takes an image obtained byperforming fluorescent observation on protein (X) localized at a cellmembrane of a cell serving as the sample 17 (hereinafter referred to ascell membrane image) and an image obtained by performing fluorescentobservation on protein (Y) localized at tTJ of the cells (hereinafterreferred to as tTJ image). Then, on the basis of the cell membrane imageand the tTJ image, the index AD of the adhesion strength of the cells iscalculated.

[Example of Configuration of Computer]

More specifically, the computer 16 of FIG. 1 is configured as shown inFIG. 2. That is, the computer 16 includes a generation unit 71, an inputunit 72, a control unit 73, and a display unit 74.

The generation unit 71 controls operation of the scan head 13, and onthe basis of the electric signal provided from the photoelectricdetection device 38 or the photoelectric detection device 41, thegeneration unit 71 generates the cell membrane image and the tTJ image,and provides them to the control unit 73. The input unit 72 isconstituted by a mouse, buttons, and the like, and provides the controlunit 73 with a signal according to user's operation.

The control unit 73 controls overall operation of the computer 16 inaccordance with the signal provided from the input unit 72. The controlunit 73 includes a cell membrane image processing unit 81, a tTJ imageprocessing unit 82, a calculation unit 83, and a processing unit 84.

The cell membrane image processing unit 81 generates, as cell analysisinformation, information about the cells such as the number of cellsincluded in the cell membrane image and the size of area of each cellfrom the cell membrane image provided from the generation unit 71. ThetTJ image processing unit 82 derives the size of area of the protein (Y)localized in the tTJ in the cell from the tTJ image provided from thegeneration unit 71.

The calculation unit 83 calculates the index AD of the adhesion strengthof the cells on the basis of the size of area of the protein (Y)localized in the tTJ of the cells and the cell analysis information. Theprocessing unit 84 processes the index AD as necessary, provides it tothe display unit 74, and displays it thereon. The display unit 74 isconstituted by, for example, a liquid crystal display panel and thelike, and displays the image and the like provided by the control unit73.

[Explanation about Operation of Observation System]

Subsequently, operation of the observation system of FIG. 1 will beexplained.

For example, when the sample 17 is observed, the epithelium cells areput in a container such as a microwell plate as the sample 17, anddifferent medicine is added to each well containing the epitheliumcells. For example, different siRNA (genes inhibiting generation ofkinase (enzyme)) is added as medicine to each well, whereby sampleshaving different observation conditions for each of the wells areprepared, e.g., the state where various kinds of kinases are knocked out(state where it is impossible to generate various kinds of kinasesrequired for phosphorylation of various kinds of proteins). Then, ineach well, the cells are observed by fluorescence, whereby the effect ofthe medicine is determined.

Such epithelium cells are prepared as the sample 17 and placed on thestage 36, and when the user commands start of observation of theepithelium cells, the observation system starts the observationprocessing, and performs the fluorescent observation of the sample 17.Hereinafter, the observation processing by the observation system willbe explained with reference to the flowchart of FIG. 3.

In step S11, the generation unit 71 obtains the cell membrane image inaccordance with the control of the control unit 73.

More specifically, the light source 11 emits illumination light. Forexample, the illumination light is light having a wavelength enablingfluorescent observation of the protein (X) localized in the cellmembrane of the cells of the observation target. The illumination lightemitted from the light source 11 passes through the beam expander 12,and through the dichroic mirror 31 to the object lens 35, and the sample17 is irradiated with the illumination light. At this occasion, the scanunit 32 deflects the illumination light, so that the illumination lightis scanned on the sample 17.

Accordingly, fluorescence is emitted from the protein (X) localized inthe cell membrane of the sample 17, and the fluorescence is incidentupon on the dichroic mirror 34 via the object lens 35, and further,after it is reflected by the dichroic mirror 34, it is condensed by thecondensing lens 37, and is received by the photoelectric detectiondevice 38. The photoelectric detection device 38 performs photoelectricconversion of the received fluorescence, and provides an electric signalobtained as a result to the generation unit 71 via the controller 15.

On the basis of the electric signal provided from the photoelectricdetection device 38, the generation unit 71 generates the cell membraneimage of the sample 17, and provides it to the control unit 73. The cellmembrane image thus obtained is an image obtained by visualizing thecells of the observation target, and the generation unit 71 generatesthe cell membrane image for each well.

In step S12, the generation unit 71 obtains the tTJ image in accordancewith the control of the control unit 73.

More specifically, the light source 11 emits illumination light toirradiate the sample 17 via the beam expander 12, and via the dichroicmirror 31 to the object lens 35. In this case, the illumination light islight having a wavelength different from that when the cell membraneimage is obtained, which enables fluorescent observation of the protein(Y) localized at a portion where mainly three cells are in contact witheach other (tTJ)

When the sample 17 is irradiated with the illumination light, theprotein (Y) localized in the tTJ emits fluorescence, and thisfluorescence is received by the photoelectric detection device 38 viathe object lens 35, the dichroic mirror 34, and the condensing lens 37.

Then, the electric signal obtained from the photoelectric conversion bythe photoelectric detection device 38 is provided via the controller 15to the generation unit 71, and accordingly, on the basis of the electricsignal, the generation unit 71 generates the tTJ image and provides itto the control unit 73. The tTJ image thus obtained is an image obtainedby visualizing the protein (Y) localized in the tTJ of the cells of theobservation target, and the generation unit 71 generates the tTJ imagefor each well.

With the processing of steps S11 and S12 explained above, for example,the cell membrane image CP11 and the cell membrane image CP12, and thetTJ image TP11 and the tTJ image TP12 are obtained, as illustrated inFIG. 4.

The cell membrane image CP11 and the cell membrane image CP12 are cellmembrane images of the cells observed under conditions different fromeach other, i.e., cell membrane images of wells different from eachother, as illustrated in FIG. 4.

The tTJ image TP11 is a tTJ image of the observation target (well) thatis the same as the cell membrane image CP11, and the tTJ image TP12 is atTJ image of the observation target that is the same as the cellmembrane image CP12. The cell membrane image CP11 and the tTJ image TP11are the images of the same portion of the observation target, and thecell membrane image CP12 and the tTJ image TP12 are the images of thesame portion of the observation target.

For example, in the cell membrane image CP11 and the cell membrane imageCP12, a region with diagonal lines is a region inside of one cell, and aregion without diagonal lines between the cells is a border region ofthe cells. More specifically, it is a region where the protein (X)localized at the cell membrane exists. In the tTJ image TP11 and the tTJimage TP12, a region without diagonal lines is a region where theprotein (Y) localized in the tTJ exists, and a region with diagonallines is a region (cell region) other than that.

In the tTJ image TP11, it is understood that, in the border portion ofthe cells as illustrated in the cell membrane image CP11, the protein(Y) is condensed in a portion where three cells are in contact with eachother (tTJ) in particular. As described above, it is said that, when theprotein (Y) is condensed in the tTJ, the adhesion strength of the cellsis high, and the cells are adhered strongly with each other.

In contrast, in the tTJ image TP12, it is understood that the protein(Y) localized in the portion where three cells are in contact with eachother (tTJ) leaks to the border portion of the cells as illustrated inthe cell membrane image CP12. As described above, when the localizationof the protein (Y) condensed at the tTJ changes, the adhesion strengthof the cells is reduced.

Back to the explanation of the flowchart of FIG. 3, when the cellmembrane image and the tTJ image are obtained the processing proceedsfrom step S12 to step S13.

In step S13, the cell membrane image processing unit 81 performssoftware matching and the like on the basis of the cell membrane imageprovided from the generation unit 71, and generates the mask of the cellmembrane image.

For example, the control unit 73 provides the display unit 74 with thecell membrane image, and the cell membrane image is displayed. Then, theuser operates the input unit 72 to designate, as the region which is tobe the mask, the region of the protein (X) localized in the cellmembrane on the cell membrane image displayed on the display unit 74,and designates, as the region which is not to be the mask, the regionwhere there is no protein (X), i.e., the region in the cell.

When such input operation is performed, the cell membrane imageprocessing unit 81 identifies, on the basis of the signal from the inputunit 72, the brightness pattern of the region which is to be the mask onthe cell membrane image (hereinafter referred to as mask region) and theregion which is not to be the mask (hereinafter referred to as maskexclusion region). Then, the cell membrane image processing unit 81performs segmentation of the region of the cell membrane on the basis ofsuch brightness pattern, and generates a mask image indicating a maskregion on the cell membrane image.

For example, suppose that the cell membrane image CP11 and the cellmembrane image CP12 as illustrated in FIG. 5 are processing target. InFIG. 5, the same reference numerals are given to the portionscorresponding to the case of FIG. 4, and the explanation thereabout isomitted as necessary.

In such case, first, the control unit 73 displays the cell membraneimage CP11 on the display unit 74. The user designates, as the maskregion, the region where diagonal lines are not attached in the cellmembrane image CP11, i.e., the region of any given cell membrane, anddesignates, as the mask exclusion region, the region in any given cellwith diagonal lines.

Then, the brightness patterns of the region with diagonal lines and theregion without diagonal lines on the cell membrane image CP11 areidentified, and the mask image MP11 is obtained, in which the region ofthe cell membrane designated as the mask region is shown. In the maskimage MP11, the region with diagonal lines indicates the mask region,and this mask region is a border of cells.

Further, the cell membrane image processing unit 81 subtracts the maskregion shown in the mask image MP11 from the original cell membraneimage CP11, whereby the region of a cell (the inside of the cell) isextracted from the cell membrane image CP11. For example, the cellmembrane image processing unit 81 derives the difference of the cellmembrane image CP11 and the mask image MP11, whereby the cell mask imageP11 showing the region of each cell (the inside of the cell) isgenerated. In the cell mask image P11, each of seven regions, i.e., theregion CR11 to the region CR17, indicates a region of one cell (theinside of the cell).

Subsequently, the cell membrane image processing unit 81 performs thesame processing as the case of the cell membrane image CP11 to generatethe mask image MP12 by segmentation on the cell membrane image CP12, andgenerates the cell mask image P12 from the cell membrane image CP12 andthe mask image MP12. In the cell mask image P12, each of four regions,i.e., the region CR21 to the region CR24, indicates a region of one cell(the inside of the cell).

When the mask image MP12 is generated, the user may designate the maskregion and the mask exclusion region on the cell membrane image CP12,but when the mask image MP11 is generated, the brightness patterns ofthese regions are already identified, and therefore, designationoperation by the user is not performed in particular.

The processing of the software matching explained above is described indetail in the specification of U.S. Pat. No. 7,203,360B2, for example.

Back to the flowchart of FIG. 3, after the cell mask image is generatedfor each cell membrane image, in step S14, the cell membrane imageprocessing unit 81 generates cell analysis information including thenumber of cells recognized in the cell membrane image and the size ofarea of each cell on the basis of the generated cell mask image. Thecell analysis information is generated for each cell membrane image.

For example, in the example of FIG. 5, information including the numberof regions “7” of the cell extracted from the cell mask image P11 andthe size of area of the region of these cells, i.e., each region of theregion CR11 to the region CR17 is generated as cell analysis informationof the cell mask image P11. In the explanation below, the size of areaof the region of each cell included in the cell analysis informationwill also be referred to as cell area size CA.

In step S15, the tTJ image processing unit 82 performs processing suchas software matching on the basis of the tTJ image provided from thegeneration unit 71, and generates the mask of the tTJ image. Morespecifically, the same processing as the processing of step S13 isperformed to generate the mask image indicating the region where theprotein (Y) localized in the tTJ exists in the tTJ image.

In step S16, the tTJ image processing unit 82 generates the expansionmask image by performing the expansion processing on the mask imagegenerated from the tTJ image.

For example, suppose that the tTJ image TP11 and the tTJ image TP12 asillustrated in FIG. 6 are processing target. In FIG. 6, the samereference numerals are given to the portions corresponding to the caseof FIG. 4, and the explanation thereabout is omitted as necessary.

In such case, first, the control unit 73 displays tTJ image TP11 on thedisplay unit 74. The user designates, as the mask region, the regionwhere diagonal lines are not attached in the tTJ image TP11, i.e., theregion of any given protein (Y) localized in the tTJ, and designates, asthe mask exclusion region, the region in any given cell with diagonallines.

Then, like the generation of the mask image of the cell membrane image,the brightness patterns of the mask region and the mask exclusion regionare identified, and the mask image MP21 is obtained, which indicates theregion of the protein (Y) designated as the mask region by segmentation.In the mask image MP21, the region with diagonal lines indicates themask region.

Further, the tTJ image processing unit 82 performs expansion processingon the mask image MP21, and generates the expansion mask image EP11which is obtained by expanding the mask region. For example, in theexpansion processing, the tTJ image processing unit 82 expands, with acertain ratio, the mask region in the mask image MP21, i.e., the regionwith diagonal lines. In the expansion mask image EP11, the region withdiagonal lines also indicates the mask region which is the region of theprotein (Y) localized in the tTJ.

Like the case of the tTJ image TP11, the tTJ image processing unit 82generates the mask image MP22 by segmentation on the tTJ image TP12, andfurther, generates the expansion mask image EP12 by expansion processingon the mask image MP22.

In step S17, for each of the cells included in the expansion mask image,the tTJ image processing unit 82 calculates the size of area of the maskregion included in the cell.

For example, the tTJ image processing unit 82 identifies the region ofeach cell in the expansion mask image EP11 of FIG. 6 on the basis of thecell mask image P11 shown in FIG. 5. Then, the tTJ image processing unit82 calculates the size of area of the mask region included in the regionof the cell in the expansion mask image EP11 for each of the cells,i.e., the size of area (hereinafter also referred to as protein areasize TA) of the region where the protein (Y) localized in the tTJ existswhich is indicated with diagonal lines.

In the explanation, the region of each cell in the tTJ image is derivedusing the cell mask image. Alternatively, the region of each cell may bederived from the expansion mask image and the like. The region of theprotein (Y) which is the mask region in the expansion mask image existsalong the border of the cells, and therefore, the border of the cellsare defined by connecting the mask regions in proximity, and the regionof each cell can be identified.

In step S18, the calculation unit 83 calculates the index AD of theadhesion strength of the cells on the basis of a cell area size CA ofeach cell in the cell membrane image derived by the cell membrane imageprocessing unit 81 and the protein area size TA of each cell on the tTJimage derived by the tTJ image processing unit 82.

For example, the calculation unit 83 calculates the following expression(1), and calculates the index AD for each cell included in each cellmembrane image.index AD=protein area size TA/cell area size CA  (1)

In this case, the index AD is a value obtained by dividing the proteinarea size TA by the cell area size CA, and therefore, indicates theratio of the region where the protein (Y) localized in the tTJ existswith respect to the entire region of the cell.

As described above, the more the protein (Y) is concentrated in the tTJ,the higher the adhesion strength of the cells is, and the protein (Y)leaks to the cell membrane, and the larger the size of area of theprotein (Y) thereof is, the lower the adhesion strength of the cellsbecomes. Therefore, it can be said that, the smaller the value of theindex AD of the cell is, the higher the adhesion strength of the cellsthereof is.

In the explanation, when the protein area size TA is calculated, themask region is expanded with a certain ratio in step S16. Alternatively,by performing the expansion processing, determination of the strength ofthe adhesion strength of the cells based on the index AD can be easilydone.

More specifically, in the expansion processing, the region of the maskis expanded in accordance with the size of area of the region.Therefore, the size of area of the mask region is increased by apredetermined ratio of the original size of area of the mask region.Therefore, when the region where the protein (Y) localized in the tTJexists is larger, the amount of increase of the index AD derived byperforming the expansion processing with respect to the value of theindex obtained without performing the expansion processing is higher. Inother words, in a cell with a low degree of adhesion strength, theprotein area size TA is amplified more greatly, and the differencebetween the index AD of a cell with a high degree of adhesion strengthand the index AD of a cell with a low degree of adhesion strength islarger.

It should be noted that the index of the adhesion strength of the cellsmay be any index as long as it is calculated from quantitativeinformation obtained from the cell membrane image and the tTJ image. Forexample, the value of cell area size CA-protein area size TA and thevalue of (cell area size CA-protein area size TA)/cell area size CA maybe adopted as the index of the adhesion strength of the cells. Thequantitative information obtained from the cell membrane image and thetTJ image is not limited to the size of area of the region of theprotein and the size of area of the region of the cell, but thequantitative information may be size information about the region of theprotein and size information about the region of the cell. For example,the size information of them may be any one of or a combination of thevolume, the size of area of the surface, the size of area, the totallength (the length of the entire periphery) of the contour of the regionof the protein, the total length (the length of the entire periphery) ofthe contour of each cell, and the strength value.

In step S19, the processing unit 84 processes the index AD of theadhesion strength of the cells calculated by the calculation unit 83 asnecessary, and provides the processing result to the display unit 74 todisplay the processing result thereon.

For example, as illustrated at the lower side of FIG. 7, the processingunit 84 generates a graph representing the index AD of the adhesionstrength derived for each cell, and displays it on the display unit 74,and as illustrated at the upper side of the figure, the processing unit84 processes the cell membrane image CP11 and the cell membrane imageCP12, and displays it on the display unit 74. In FIG. 7, the samereference numerals are given to the portions corresponding to the caseof FIG. 4, and the explanation thereabout is omitted.

In the graph as illustrated at the lower side of FIG. 7, the horizontalaxis denotes the name given to each cell, and the vertical axis denotesthe value of the index AD of the adhesion strength derived for each ofthe cells. The processing unit 84 assumes that a cell of which indexvalue AD of adhesion strength is more than a predetermined thresholdvalue th is a cell of which adhesion strength is low (weak), and a cellof which index value AD is less than or equal to the threshold value this a cell of which adhesion strength is high (strong).

In the graph of FIG. 7, the threshold value th is “0.0746”, and sevencells at the left side of the graph are cells having strong adhesionstrength. These cells are cells included in, for example, the cellmembrane image CP11. As a result of comparison with the threshold valueth, four cells at the right side of the graph are cells of weak adhesionstrength. These cells are cells included in, for example, the cellmembrane image CP12.

The threshold value th may be directly designated by the user, or may bederived from calculation result and the like of the index AD for eachcell. For example, the user may cause the tTJ image to be displayed, anddesignate cells of which adhesion strength is considered to be weak andcells of which adhesion strength is considered to be strong, and thethreshold value th may be calculated from the indexes AD of the cellsdesignated by the user.

Further, processing unit 84 processes the cell membrane image CP11 andthe cell membrane image CP12 on the basis of the comparison result (theresult of the threshold value processing) of the index AD and thethreshold value th.

For example, as shown at the upper side in the figure, the processingunit 84 encircles each cell with a frame in the cell membrane imageCP11, and displays the region in the frame in a display format accordingto the comparison result of the index AD and the threshold value th. Inthe cell membrane image CP11, each of the rectangular region W11 to therectangular region W17 including each cell is displayed in a colorindicating that each cell has high adhesion strength (for example, red).

Likewise, the processing unit 84 encircles each cell with a frame in thecell membrane image CP12, and displays the region in the frame in adisplay format according to the comparison result of the index AD andthe threshold value th. In the cell membrane image CP12, each of therectangular region W21 to the rectangular region W24 including each cellis displayed in a color indicating that each cell has low adhesionstrength (for example, blue).

In addition to the processing result as shown in FIG. 7, as shown inFIG. 8, the processing unit 84 may generate statistics informationindicating strength determination result of adhesion of the cellsincluded in the well for each well (observation condition), and maydisplay the statistics information on the display unit 74.

In FIG. 8, the horizontal axis denotes the name given to each well, andthe vertical axis denotes a ratio of a cell having high adhesionstrength and a cell having low adhesion strength in each well.

For example, each of A1 to H12 as shown in the horizontal axisrepresents the name given to each well. The bar graph without diagonallines in each well represents a ratio of the number of cells determinedto have high adhesion strength with respect to the total number of cellsincluded in the well. The bar graph with diagonal lines in each wellrepresents a ratio of the number of cells determined to have lowadhesion strength with respect to the total number of cells included inthe well.

In the example of FIG. 8, in most wells, there are fewer cells havinglow adhesion strength, and most of the cells are cells having highadhesion strength, but in the well “H10”, there are fewer cells havinghigh adhesion strength, and most of the cells are cells having lowadhesion strength. It is understood from this result that the effect ofmedicine is achieved under the observation condition of the well “H10”.

It should be noted that the threshold value may be set by user'sdesignation and the like, and when the ratio of the cells having lowadhesion strength is equal to or more than the threshold value, theprocessing unit 84 may determine presence of the effect of the medicine.In this case, presence/absence of the effect of the medicine in eachwell can be quantitatively determined.

When the processing unit 84 processes and outputs the index AD of theadhesion strength as described above, the observation processing isfinished.

As described above, the observation system obtains the cell membraneimage and the tTJ image, derives the cell area size CA and the proteinarea size TA from these images, and derives the ratio as the index AD ofthe adhesion strength of each cell.

In the present embodiment, the cell membrane image and the tTJ image areobtained, and the cell area size CA and the protein area size TA areobtained from these images, but the present embodiment is not limitedthereto. For example, the region where the protein (Y) localized in thetTJ exists may be identified from the tTJ image, and the region of thecell may be predicted (using publicly-known prediction means) andidentified, and the cell area size CA and the protein area size TA maybe derived, and the index AD of the adhesion strength may be calculated.

As described above, by deriving the index AD of the adhesion strength ofeach cell, the feature of particular cells, e.g., the degree of adhesionof the cells, can be quantitatively observed.

By processing the index AD of the adhesion strength derived for eachcell as necessary and presenting the processing result thereof, it ispossible to easily identify a condition under which the effect of themedicine can be achieved. More specifically, by analyzing individualcells with certain criteria, the effect of the medicine during medicinescreening can be determined more efficiently. In particular, as comparedto determination based on visual inspection which has been done in thepast, determination and the like can be done objectively in a shorttime.

In the above explanation, for example, localization of the proteinrelated to the adhesion strength of the cells is observed, and the indexrepresenting the adhesion strength is calculated as an index indicatingthe feature of the cell. Alternatively, a value representing the degreeof activity of the cell may be calculated.

The series of processing explained above may be executed either byhardware or software. When the above series of processing is executed bysoftware, a program constituting the software is read and recorded froma recording medium by, for example, the computer 16. Then, the programrecorded in the computer 16 is executed by the computer 16, and theobservation processing and the like of FIG. 3 is executed.

The embodiment of the present invention is not limited to the embodimentexplained above. It may be changed in various manners without deviatingfrom the gist of the present invention.

REFERENCE SIGNS LIST

-   -   13 scan head, 14 microscope, 16 computer, 17 sample, 71        generation unit, 73 control unit, 74 display unit, 81 cell        membrane image processing unit, 82 tTJ image processing unit, 83        calculation unit, 84 processing unit

The invention claimed is:
 1. An image processing apparatus comprising: aprocessor configured to: calculate quantitative information related to aregion of at least one protein in a first observation image, andcalculate, based on the calculated quantitative information related tothe region of the at least one protein, an index representing anadhesion strength of at least one cell.
 2. The image processingapparatus according to claim 1, wherein the processor is furtherconfigured to: calculate quantitative information related to a region ofthe at least one cell, and calculate on the basis of the calculatedquantitative information related to the region of the at least oneprotein, and the calculated quantitative information related to theregion of the at least one cell, the index representing the adhesionstrength of the at least one cell.
 3. The image processing apparatusaccording to claim 2, wherein the processor is further configured tocalculate the index representing the adhesion strength of the at leastone cell by calculating a ratio of the calculated quantitativeinformation related to the region of the at least one protein and thecalculated quantitative information related to the region of the atleast one cell.
 4. The image processing apparatus according to claim 1,wherein the first observation image is an observation image having theat least one protein localized in a tricellular tight junction of the atleast one cell.
 5. The image processing apparatus according to claim 1,wherein the processor is further configured to determine whether theadhesion strength of the at least one cell is strong or weak based onthe index representing the adhesion strength of the at least one cell.6. The image processing apparatus according to claim 5, wherein theprocessor is further configured to determine whether the adhesionstrength of the at least one cell is strong or weak by comparing theindex representing the adhesion strength of the at least one cell and athreshold value.
 7. The image processing apparatus according to claim 5,wherein the processor is further configured to control a display todisplay information related to a number of cells determined to havestrong adhesion strength and/or a number of cells determined to haveweak adhesion strength.
 8. The image processing apparatus according toclaim 5, wherein the processor is further configured to determine aneffect of medicine based on information related to a number of cellsdetermined to have strong adhesion strength and/or a number of cellsdetermined to have weak adhesion strength.
 9. The image processingapparatus according to claim 1, wherein the processor is furtherconfigured to: generate a graph representing the index of the adhesionstrength of the at least one cell, and control a display to display thegenerated graph representing the adhesion strength of the at least onecell.
 10. The image processing apparatus according to claim 2, whereinthe processor is further configured to calculate the quantitativeinformation related to the region of the at least one cell based on asecond observation image.
 11. The image processing apparatus accordingto claim 10, wherein the first observation image is an image capturedwith irradiation of a light having a first wavelength on the at leastone protein, and the second observation image is an image captured withirradiation of a light having a second wavelength on the at least onecell.
 12. A microscope system comprising: an object lens; an imagingapparatus configured to capture a plurality of well images of aplurality of wells; and a processor configured to: for each well imageof the plurality of well images captured by the imaging apparatus:calculate quantitative information related to a region of at least oneprotein in the well image, calculate an index representing an adhesionstrength of at least one cell in the well image based on the calculatedquantitative information related to the region of the at least oneprotein in the well image, and determine an effect of medicine based onthe index representing the adhesion strength of the at least one cell inthe well image.
 13. A microscope system comprising: an object lens; animaging apparatus configured to capture a plurality of well images of aplurality of wells; a display; and a processor configured to: for eachwell image of the plurality of well images captured by the imagingapparatus: calculate quantitative information related to a region of atleast one protein in the well image, calculate an index representing anadhesion strength of at least one cell in the well image based on thecalculated quantitative information related to the region of the atleast one protein in the well image, determine whether the adhesionstrength of the at least one cell in the well image is strong or weakbased on the index representing the adhesion strength of the at leastone cell in the well image, and control the display to displayinformation related to a number of cells in the well image determined tohave strong adhesion strength and/or a number of cells in the well imagedetermined to have weak adhesion strength.
 14. The microscope systemaccording to claim 13, wherein the processor is further configured to,for each well image, determine an effect of medicine based on theinformation related to the number of cells of the well image determinedto have strong adhesion strength and/or the number of cells of the wellimage determined to have weak adhesion strength.
 15. An image processingapparatus comprising: a processor configured to: calculate informationrelated to a size of a region of at least one protein in a firstobservation image, and calculate, based on the calculated informationrelated to the size of the region of the at least one protein, an indexrepresenting adhesion strength of at least one cell.
 16. The imageprocessing apparatus according to claim 15, wherein the processor isfurther configured to: calculate information related to a size of aregion of the at least one cell, and calculate, based on the calculatedinformation related to the size of the region of the at least oneprotein and the calculated information related to the size of the regionof the at least one cell, the index representing the adhesion strengthof the at least one cell.
 17. The image processing apparatus accordingto claim 16, wherein the processor is further configured to calculatethe index representing the adhesion strength of the at least one cell bycalculating a ratio of the calculated information related to the size ofthe region of the at least one protein and the calculated informationrelated to the size of the region of the at least one cell.
 18. Theimage processing apparatus according to claim 15, wherein the firstobservation image is an observation image having the at least oneprotein localized in a tricellular tight junction of the at least onecell.
 19. The image processing apparatus according to claim 15, whereinthe processor is further configured to determine whether the adhesionstrength of the at least one cell is strong or weak based on the indexrepresenting the adhesion strength of the at least one cell.
 20. Theimage processing apparatus according to claim 19, wherein the processoris further configured to determine whether the adhesion strength of thecell is strong or weak by comparing the index representing the adhesionstrength of the at least one cell and a threshold value.
 21. The imageprocessing apparatus according to claim 19, wherein the processor isfurther configured to control a display to display information relatedto a number of cells determined to have strong adhesion strength and/ora number of cells determined to have weak adhesion strength.
 22. Theimage processing apparatus according to claim 19, wherein the processoris further configured to determine an effect of medicine based oninformation related to a number of cells determined to have strongadhesion strength and/or a number of cells determined to have weakadhesion strength.
 23. The image processing apparatus according to claim15, wherein the processor is further configured to: generate a graphrepresenting the index of the adhesion strength of the at least onecell, and control a display to display the generated graph representingthe adhesion strength of the at least one cell.
 24. The image processingapparatus according to claim 16, wherein the processor is furtherconfigured to calculate the information related to the size of theregion of the at least one cell based on a second observation image. 25.The image processing apparatus according to claim 24, wherein the firstobservation image is an image captured with irradiation of a lighthaving a first wavelength on the at least one protein, and the secondobservation image is an image captured with irradiation of a lighthaving a second wavelength on the at least one cell.
 26. A microscopesystem comprising: an object lens; an imaging apparatus configured tocapture a plurality of well images of a plurality of wells; and aprocessor configured to: for each well image of the plurality of wellimages captured by the imaging apparatus: calculate information relatedto a size of a region of at least one protein in the well image,calculate an index representing an adhesion strength of at least onecell in the well image based on the calculated information related tothe size of the region of the at least one protein in the well image,and determine an effect of medicine based on the index representing theadhesion strength of the at least one cell in the well image.
 27. Amicroscope system comprising: an object lens; an imaging apparatusconfigured to capture a plurality of well images of a plurality ofwells; a display; and a processor configured to: for each well image ofthe plurality of well images captured by the imaging apparatus:calculate information related to a size of a region of at least oneprotein in the well image, calculate an index representing an adhesionstrength of at least one cell in the well image based on the calculatedinformation related to the size of the region of the at least oneprotein in the well image, determine whether the adhesion strength ofthe at least one cell in the well image is strong or weak based on theindex representing the adhesion strength of the at least one cell in thewell image, and control the display to display information related to anumber of cells in the well image determined to have strong adhesionstrength and/or a number of cells determined to have weak adhesionstrength.
 28. The microscope system according to claim 27, wherein theprocessor is configured to, for each well image, determine an effect ofmedicine based on the information related to the number of cells of thewell image determined to have strong adhesion strength and/or the numberof cells of the well image determined to have weak adhesion strength.29. An image processing apparatus comprising: a processor configured to:calculate information related to an area of at least one protein in afirst observation image, and calculate, based on the calculatedinformation related to the area of the at least one protein, an indexrepresenting adhesion strength of at least one cell.
 30. The imageprocessing apparatus according to claim 29, wherein the processor isfurther configured to: calculate information related to an area of theat least one cell, and calculate, based on the calculated informationrelated to the area of the at least one protein, and the calculatedinformation related to the area of the at least one cell, the indexrepresenting the adhesion strength of the at least one cell.
 31. Theimage processing apparatus according to claim 30, wherein the processoris further configured to calculate the index representing the adhesionstrength of the at least one cell by calculating a ratio of thecalculated information related to the area of the at least one proteinand the calculated information related to the area of the at least onecell.
 32. The image processing apparatus according to claim 29, whereinthe first observation image is an observation image having the at leastone protein localized in a tricellular tight junction of the at leastone cell.
 33. The image processing apparatus according to claim 29,wherein the processor is further configured to determine whether theadhesion strength of the at least one cell is strong or weak based onthe index representing the adhesion strength of the at least one cell.34. The image processing apparatus according to claim 33, wherein theprocessor is further configured to determine whether the adhesionstrength of the at least one cell is strong or weak by comparing theindex representing the adhesion strength of the at least one cell and athreshold value.
 35. The image processing apparatus according to claim33, wherein the processor is further configured to control a display todisplay information related to a number of cells determined to havestrong adhesion strength and/or a number of cells determined to haveweak adhesion strength.
 36. The image processing apparatus according toclaim 33, wherein the processor is further configured to determine aneffect of medicine based on information related to a number of cellsdetermined to have strong adhesion strength and/or a number of cellsdetermined to have weak adhesion strength.
 37. The image processingapparatus according to claim 29, wherein the processor is furtherconfigured to: generate a graph representing the index of the adhesionstrength of the at least one cell, and control a display to display thegenerated graph representing the adhesion strength of the at least onecell.
 38. The image processing apparatus according to claim 30, whereinthe processor is further configured to calculate the information relatedto the area of the at least one cell based on a second observationimage.
 39. The image processing apparatus according to claim 38, whereinthe first observation image is an image captured with irradiation of alight having a first wavelength on the at least one protein, and thesecond observation image is an image captured with irradiation of alight having a second wavelength on the at least one cell.
 40. Amicroscope system comprising: an object lens; an imaging apparatusconfigured to capture a plurality of well images of a plurality ofwells; and a processor configured to: for each well image of theplurality of well images captured by the imaging apparatus: calculateinformation related to an area of at least one protein in the wellimage, calculate an index representing an adhesion strength of at leastone cell in the well image based on the calculated information relatedto the area of the at least one protein in the well image, and determinean effect of medicine based on the index representing the adhesionstrength of the at least one cell in the well image.
 41. A microscopesystem comprising: an object lens; an imaging apparatus configured tocapture a plurality of well images of a plurality of wells; a display;and a processor configured to: for each well image of the plurality ofwell images captured by the imaging apparatus: calculate informationrelated to an area of at least one protein in the well image, calculatean index representing an adhesion strength of at least one cell in thewell image based on the calculated information related to the area ofthe at least one protein in the well image, determine whether theadhesion strength of the at least one cell in the well image is strongor weak based on the index representing the adhesion strength of the atleast one cell in the well image, and control the display to displayinformation related to a number of cells in the well image determined tohave strong adhesion strength and/or a number of cells in the well imagedetermined to have weak adhesion strength.
 42. The microscope systemaccording to claim 41, wherein the processor is further configured to,for each well image, determine an effect of medicine based on theinformation related to the number of cells in the well image determinedto have strong adhesion strength and/or the number of cells in the wellimage determined to have weak adhesion strength.